WIP: Use a well known 9p port range instead of random port #21275
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The --mount-string argument defaults to `/Users` on darwin, and
homedir.Homedir() on other platforms (e.g. $HOME on unix).
This is wrong in many ways:
- `/Users` is not HOME on darwin (the right path is `/Users/$USER`).
Using the default mount we cannot access anything inside the guest in
the user home directory. We can access the special `/Users/Shared`
directory, but this should not be a default mount.
- Mounting the user home directory inside the guest in read-write mode
is a horrible default. This exposes the users private keys in .ssh/ to
the guest, any sensitive files in the user home directory, and allows
the guest to change any file on the host.
- Using the `--mount` option mount the default mount directory silently.
This is unexpected, surprising, and not documented in the minikube
handbook[1].
Example access to user private key from the guest with the default
mount:
$ minikube start --mount
$ minikube ssh cat /minikube-host/.ssh/id_ed25519
-----BEGIN OPENSSH PRIVATE KEY-----
...
-----END OPENSSH PRIVATE KEY-----
Fixed by removing the default mount directory and changing mount logic
to check for non-empty mount-string instead of the mount flag.
The mount flag is kept for backward compatibility, but its value is
ignored. In the next release we want to use this flag for supporting
multiple mounts.
Example usage before:
minikube start --mount --mount-string ~/models:/mnt/models
Example usage after:
minikube start --mount-string ~/models:/mnt/models
Breaking changes:
User depending the default mount will have to replace the command:
minikube start --mount
With:
minikube start --mount-string $HOME:/minikube-host
[1] https://minikube.sigs.k8s.io/docs/handbook/mount/
The mount tests on Fedora fail with: $ out/minikube mount /tmp/mount:/minikube-host 📁 Mounting host path /tmp/mount into VM as /minikube-host ... ▪ Mount type: 9p ▪ User ID: docker ▪ Group ID: docker ▪ Version: 9p2000.L ▪ Message Size: 262144 ▪ Options: map[] ▪ Bind Address: 192.168.39.1:43405 🚀 Userspace file server: ufs starting ❌ Exiting due to GUEST_MOUNT: mount failed: mount with cmd /bin/bash -c "sudo mount -t 9p -o dfltgid=$(grep ^docker: /etc/group | cut -d: -f3),dfltuid=$(id -u docker),msize=262144,port=43405,trans=tcp,version=9p2000.L 192.168.39.1 /minikube-host" : /bin/bash -c "sudo mount -t 9p -o dfltgid=$(grep ^docker: /etc/group | cut -d: -f3),dfltuid=$(id -u docker),msize=262144,port=43405,trans=tcp,version=9p2000.L 192.168.39.1 /minikube-host": Process exited with status 32 stdout: stderr: mount: /minikube-host: fsconfig system call failed: Connection refused. dmesg(1) may have more information after failed mount system call. The reason for the failure is the firewall in Fedora allows only certain services on the libvirt zone: $ sudo firewall-cmd --zone libvirt --list-all libvirt (active) target: ACCEPT ingress-priority: 0 egress-priority: 0 icmp-block-inversion: no interfaces: virbr0 sources: services: dhcp dhcpv6 dns ssh tftp ports: protocols: icmp ipv6-icmp forward: no masquerade: no forward-ports: source-ports: icmp-blocks: rich rules: rule priority="32767" reject We can fix 9p mounts by opening a port range for 9p and using this port range in the tests: $ sudo firewall-cmd --zone libvirt --add-port 9000-9100/tcp --permanent success $ sudo firewall-cmd --zone libvirt --list-ports 9000-9100/tcp $ out/minikube mount /tmp:/minikube-host --port 9000 📁 Mounting host path /tmp into VM as /minikube-host ... ▪ Mount type: 9p ▪ User ID: docker ▪ Group ID: docker ▪ Version: 9p2000.L ▪ Message Size: 262144 ▪ Options: map[] ▪ Bind Address: 192.168.39.1:9000 🚀 Userspace file server: ufs starting ✅ Successfully mounted /tmp to /minikube-host 📌 NOTE: This process must stay alive for the mount to be accessible ... This change only fixes the port in the test. Missing changes: - change all mount tests (only TestMountStart fixed) - change the dfault port in --mount-port= - document the port range and how to open ports in the firewall - add default-9p-port-range to the config so users can change it if it clashes with another service - use the default port range in the tests
|
Skipping CI for Draft Pull Request. |
k8s-ci-robot
added
the
do-not-merge/work-in-progress
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: nirs The full list of commands accepted by this bot can be found here.
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
k8s-ci-robot
added
cncf-cla: yes
nirs
changed the title
k8s-ci-robot
added
the
needs-rebase
|
PR needs rebase. Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
|
This should really be an issue and much more work is needed. Will reopen when I have time to work on this and we have agreement on the design. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Using a random port is not practical when using a firewall blocking access to the host on the libvirt network, which is the default in Fedora (for good reason).
We want to switch to well known port range so users can configure their firewall. User that want to reproduce the old way we configure the port range to similar port range (46463-49152). They can also select the port using --mount-port.
Minikube should manage the mount ports by keeping a pool of ports and picking the next available port from the pool.
Based on #21250 for testing